Image predictive decoding method, image predictive decoding apparatus, image predictive coding method, image predictive coding apparatus, and data storage media

ABSTRACT

Disclosed is an image predictive decoding method in which image data obtained by compressively coding a variable-size image using a prescribed method is input, a prediction image is generated using, as a reference image, at least one reproduced image which has been reproduced before an image being an object of decoding, and the object image is subjected to predictive decoding. In this method, the prediction image is generated using, as a reference image, at least one reproduced image which has been recently reproduced and includes significant image data to be referred to. Therefore, when plural objects constituting an image are subjected to compressive coding and transmitted object by object to increase the compression efficiency, it is avoided that a variable-size image which has already disappeared is used as the reference image for predictive decoding. As a result, coded data obtained by efficient compressive coding that suppresses the code quantity can be appropriately decoded.

This is a Rule 1.53(b) Continuation Application of Ser. No. 09/054,503filed Apr. 3, 1998, now pending.

FIELD OF THE INVENTION

The present invention relates to image predictive decoding and imagepredictive coding and, more particularly, to image predictive decodingmethods, image predictive decoding apparatuses, image predictive codingmethods, image predictive coding apparatuses, and data storage media,which are used for processing variable-size images.

BACKGROUND OF THE INVENTION

In order to store or transmit a digital image with high efficiency, itis necessary to compressively code the digital image. As a typicalmethod for compressive coding of a digital image, there is DCT (DiscreteCosine Transformation) represented by JPEG (Joint Photographic ExpertsGroup) and MPEG (Moving Picture Experts Group). Besides, there arewaveform coding methods such as sub-band coding, wavelet coding, andfractal coding. Further, in order to eliminate a redundant signalbetween images, inter-image prediction using motion compensation iscarried out, and a difference signal is subjected to waveform coding.

Here, an MPEG method based on motion compensation DCT will be described.Initially, an input image of one frame to be coded is divided intoplural macroblocks each having the size of 16×16 pixels. Each macroblockis further divided into four blocks each having the size of 8×8 pixels,and each block of 8×8 pixels is subjected to DCT and quantization. Thisprocess is called “intra-frame coding”.

On the other hand, using a motion detecting method such as blockmatching, from a frame temporarily adjacent to an object frame includingan object macroblock to be quantized, a prediction macroblock having thesmallest error from the object macroblock is detected, and motioncompensation from the past image is carried out on the basis of thedetected motion, thereby to obtain an optimum prediction block. A signalshowing the motion toward the prediction macroblock having the smallesterror is a motion vector. An image used as a reference for generatingthe prediction macroblock is called a reference image, hereinafter.Thereafter, a difference between the object block and the correspondingprediction block is obtained, and this difference is subjected to DCT toobtain a DCT coefficient. The DCT coefficient is quantized, and thequantized output is transmitted or stored together with the motioninformation. This process is called “inter-frame coding”.

The inter-frame coding has two prediction modes: prediction from aprevious image in the display order, and prediction from both ofprevious and future images. The former is called “forward prediction”,and the latter is called “bidirectional prediction”.

On the decoder end, after restoring the quantized DCT coefficient to theoriginal difference signal, the prediction block is obtained on thebasis of the difference signal and the motion vector, and the predictionblock and the difference signal are added to reproduce the image. Inthis conventional technique, it is premised that the size of thereference image (an image used as a reference for generating aprediction image) is equal to the size of the object image.

In recent years, plural objects constituting an image (arbitrary shapeimages) are separately subjected to compressive coding and transmitted,thereby to improve the coding efficiency and to enable object by objectreproduction. In coding and decoding of such arbitrary shape image, thesize of the image changes very often. For example, a ball becomessmaller and smaller, till at last it disappears. Further, there is acase where the size of the image (object) becomes zero.

In ordinary predictive coding, a reference image is a reproduced imagejust before an object image which is currently being processed. When thesize of the reference image is zero, since nothing is defined in thereference image, i.e., since the reference image has no significantimage content data to be used for predictive coding, predictive codingcannot be carried out. In this case, there is no conventional way exceptthe intra-frame coding. However, generally the intra-frame codingincreases the quantity of coded data and reduces the compressionefficiency. When an image disappears (image size=zero) and appearsfrequently in a sequence of motion picture, the coding efficiency issignificantly degraded. For example, in a motion picture of flashingspotlight, when the light disappears and appears in image units, all theimages of lights must be subjected to the intra-frame coding.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide an image predictivedecoding method, an image predictive decoding apparatus, an imagepredictive coding method, an image predictive coding apparatus, and adata storage medium, which can realize efficient predictive coding ordecoding of a variable-size image even when the size of a referenceimage is zero or when the reference image is completely transparent.

Other objects and advantages of the invention will become apparent fromthe detailed description that follows. The detailed description andspecific embodiments described are provided only for illustration sincevarious additions and modifications within the scope of the inventionwill be apparent to those of skill in the art from the detaileddescription.

According to a first aspect of the present invention, there is providedan image predictive decoding method in which image data obtained bycompressively coding a variable-size image using a prescribed method isinput, a prediction image is generated using, as a reference image, atleast one reproduced image which has been reproduced before an imagebeing an object of decoding, and the object image is subjected topredictive decoding; wherein the prediction image is generated using, asa reference image, at least one reproduced image which has been recentlyreproduced and includes significant image content data to be referredto. In this method, when the size of the reference image (reproducedimage) is zero, i.e., when the reference image is completelytransparent, predictive decoding is carried out using another reproducedimage of which size is not zero. Therefore, when plural objectsconstituting an image are subjected to compressive coding andtransmitted object by object to increase the compression efficiency, itis avoided that a variable-size image which has already disappeared isused as the reference image for predictive decoding. As a result, codeddata obtained by efficient compressive coding that suppresses the codequantity can be appropriately decoded.

According to a second aspect of the present invention, there is providedan image predictive decoding method in which image data obtained bycompressively coding a variable-size image using a prescribed method isinput, a prediction image is generated using, as a reference image, aprescribed image which has been reproduced before an image being anobject of decoding, and the object image is subjected to predictivedecoding; wherein, when the reproduced image used as a reference imagehas no significant coded data to be referred to, an image having aprescribed value as its image data is used as the prediction image. Inthis method, when the size of the reference image (reproduced image) iszero, i.e., when the reference image is completely transparent,predictive decoding is carried out using a prediction image having aprescribed value. Therefore, in addition to the above-mentioned effects,generation of the prediction image is facilitated.

According to a third aspect of the present invention, there is providedan image predictive decoding method in which image data obtained bycompressively coding a variable-size image using a prescribed method isinput, a prediction image is generated using a reference image, and animage being an object of decoding is subjected to predictive decoding;wherein the prediction image is generated using, as the reference image,at least one of two reproduced signals which have been recentlyreproduced, which one has significant image data to be referred to.Therefore, in the case where plural objects constituting an image aresubjected to compressive coding and transmitted object by object, when aprediction image is generated using forward and backward referenceimages, it is avoided that variable-size images which have alreadydisappeared are used as the reference images. As a result, coded dataobtained by efficient compressive coding that suppresses the codequantity can be appropriately decoded.

According to a fourth aspect of the present invention, there is providedan image predictive decoding apparatus comprising input means to whichimage data obtained by compressively coding a variable-size image usinga prescribed method is applied; a data analyzer which analyzes the imagedata and outputs the image size and the image transform coefficient; adecoder which restores the image transform coefficient to an expandeddifference image using a prescribed method; a frame memory that containsa reproduced image; a prediction image generator that generates aprediction image using, as a reference image, the reproduced imagestored in the frame memory; and an adder that generates a reproducedimage by adding the expanded difference image and the prediction image,and outputs the reproduced image and, simultaneously, stores thereproduced image into the frame memory; wherein the prediction imagegenerator examines whether or not the reproduced image includessignificant image data to be referred to, and generates a predictionimage which has been recently reproduced and includes significant imagedata. In this apparatus, when the size of the reference image(reproduced image) is zero, i.e., when the reference image is completelytransparent, predictive decoding is carried out using another reproducedimage of which size is not zero. Therefore, when plural objectsconstituting an image are subjected to compressive coding andtransmitted object by object to increase the compressive efficiency, itis avoided that a variable-size image which has already disappeared isused as the reference image for predictive decoding. As a result, codeddata obtained by efficient compressive coding that suppresses the codequantity can be appropriately decoded.

According to a fifth aspect of the present invention, there is providedan image predictive decoding apparatus comprising input means to whichimage data obtained by compressively coding a variable-size image usinga prescribed method is applied; a data analyzer which analyzes the imagedata and outputs the image size and the image transform coefficient; adecoder which restores the image transform coefficient to an expandeddifference image using a prescribed method; a frame memory that containsa reproduced image; a prediction image generator that generates aprediction image using, as a reference image, a prescribed reproducedimage stored in the frame memory and corresponding to the input image;and an adder that generates a reproduced image by adding the expandeddifference image and the prediction image, and outputs the reproducedimage and, simultaneously, stores the reproduced image into the framememory; wherein the prediction image generator examines whether or notthe prescribed reproduced image has significant image data to bereferred to, and when the reproduced image has no significant coded datato be referred to, an image having a prescribed value as its image datais used as the prediction image. In this apparatus, when the size of thereference image (reproduced image) is zero, i.e., when the referenceimage is completely transparent, predictive decoding is carried outusing a prediction image having a prescribed value. Therefore, inaddition to the above-mentioned effects, generation of the predictionimage is facilitated.

According to a sixth aspect of the present invention, there is providedan image predictive decoding apparatus comprising input means to whichimage data obtained by compressively coding a variable-size image usinga prescribed method is applied; a data analyzer which analyzes the imagedata and outputs the image size and the image transform coefficient; adecoder which restores the image transform coefficient to an expandeddifference image using a prescribed method; a frame memory that containsa reproduced image; a prediction image generator that generates aprediction image using the reproduced image stored in the frame memoryas a reference image; and an adder that generates a reproduced image byadding the expanded difference image and the prediction image, andoutputs the reproduced image and, simultaneously, stores the reproducedimage into the frame memory; wherein the prediction image generatorgenerates the prediction image using, as the reference image, at leastone of two reproduced signals which have been recently reproduced, whichone has significant image data to be referred to. Therefore, in the casewhere plural objects constituting an image are subjected to compressivecoding and transmitted object by object, when a prediction image isgenerated using forward and backward reference images, it is avoidedthat variable-size images which have already disappeared are used as thereference images. As a result, coded data obtained by efficientcompressive coding that suppresses the code quantity can beappropriately decoded.

According to a seventh aspect of the present invention, there isprovided an image predictive coding method in which a variable-sizeimage is input, a prediction image is generated using, as a referenceimage, at least one reproduced image which has been reproduced before animage being an object of coding, the object image is subtracted from theprediction image, and a difference between these images is compressivelycoded by a prescribed method; wherein the prediction image is generatedusing, as a reference image, at least one reproduced image which hasbeen recently reproduced and includes significant image data to bereferred to. In this method, when the size of the reference image(reproduced image) is zero, i.e., when the reference image is completelytransparent, predictive coding is carried out using another reproducedimage of which size is not zero. Therefore, when plural objectsconstituting an image are subjected to compressive coding andtransmitted object by object to increase the compression efficiency, itis avoided that a variable-size image which has already disappeared isused as the reference image for predictive coding, resulting in apredictive coding method capable of efficient compressive coding thatsuppresses the code quantity.

According to an eighth aspect of the present invention, there isprovided an image predictive coding method in which a variable-sizeimage is input, a prediction image is generated using, as a referenceimage, a prescribed reproduced image which has been reproduced before animage being an object of coding, the object image is subtracted from theprediction image, and a difference between these images is compressivelycoded by a prescribed method; wherein, when the reproduced image used asa reference image has no significant image data to be referred to, animage having a prescribed value as its image data is used as theprediction image. In this method, when the size of the reference image(reproduced image) is zero, i.e., when the reference image is completelytransparent, predictive coding is carried out using a prediction imagehaving a prescribed value. Therefore, in addition to the above-mentionedeffects, generation of the prediction image is facilitated.

According to a ninth aspect of the present invention, there is providedan image predictive coding method in which a variable-size image isinput, a prediction image is generated using a reference image, anobject image being an object of coding is subtracted from the predictionimage, and a difference between these images is compressively coded by aprescribed method; wherein the prediction image is generated using, asthe reference image, at least one of two reproduced images which hasbeen recently reproduced and includes significant image data to bereferred to. Therefore, in the case where plural objects constituting animage are subjected to compressive coding and transmitted object byobject, when a prediction image is generated using forward and backwardreference images, it is avoided that variable-size images which havealready disappeared are used as the reference images, resulting in apredictive coding method capable of efficient compressive coding thatsuppresses the code quantity.

According to a tenth aspect of the present invention, there is providedan image predictive coding apparatus comprising input means to whichdata of a variable-size image is input, which data is divided into unitssubjected to coding; a subtracter that obtains a difference imagebetween an object image being an object of coding and a prediction imagecorresponding to the object image; a compressive encoder that convertsthe difference image to compressed data by a prescribed compressivecoding process; a variable-length encoder that performs variable-lengthcoding of the compressed data and outputs coded data; an expansivedecoder that restores the compressed data to an expanded differenceimage by a prescribed expansive decoding process; a frame memory thatcontains a reproduced image; a prediction image generator that generatesa prediction image using the reproduced image stored in the frame memoryas a reference image; and an adder that generates a reproduced image byadding the expanded difference image and the prediction image, andoutputs the reproduced image and, simultaneously, stores the reproducedimage into the frame memory; wherein the prediction image generatorexamines whether or not the reproduced image has significant image datato be referred to, and generates the prediction image using, as areference image, at least one reproduced image which has been recentlyreproduced and includes significant image data. In this apparatus, whenthe size of the reference image (reproduced image) is zero, i.e., whenthe reference image is completely transparent, predictive coding iscarried out using another reproduced image of which size is not zero.Therefore, when plural objects constituting an image are subjected tocompressive coding and transmitted object by object to increase thecompression efficiency, it is avoided that a variable-size image whichhas already disappeared is used as the reference image for predictivecoding, resulting in a predictive coding apparatus capable of efficientcompressive coding that suppresses the code quantity.

According to an eleventh aspect of the present invention, there isprovided an image predictive coding apparatus comprising input means towhich data of a variable-size image is input, which data is divided intounits subjected to coding; a subtracter that obtains a difference imagebetween an object image being an object of coding and a prediction imagecorresponding to the object image; a compressive encoder that convertsthe difference image to compressed data by a prescribed compressivecoding process; a variable-length encoder that performs variable-lengthcoding of the compressed data and outputs coded data; an expansivedecoder that restores the compressed data to an expanded differenceimage by a prescribed expansive decoding process; a frame memory thatcontains a reproduced image; a prediction image generator that generatesa prediction image using the reproduced image stored in the frame memoryas a reference image; and an adder that generates a reproduced image byadding the expanded difference image and the prediction image, andoutputs the reproduced image and, simultaneously, stores the reproducedimage into the frame memory; wherein the prediction image generatorexamines whether or not the reproduced image has significant image datato be referred to and, when the reproduced image has no significantimage data, an image having a prescribed value as its image data is usedas the prediction image. In this apparatus, when the size of thereference image (reproduced image) is zero, i.e., when the referenceimage is completely transparent, predictive coding is carried out usinga prediction image having a prescribed value. Therefore, in addition tothe above-mentioned effects, generation of the prediction image isfacilitated.

According to a twelfth aspect of the present invention, there isprovided an image predictive coding apparatus comprising input means towhich data of a variable-size image is input, which data is divided intounits subjected to coding; a subtracter that obtains a difference imagebetween an object image being an object of coding and a prediction imagecorresponding to the object image; a compressive encoder that convertsthe difference image to compressed data by a prescribed compressivecoding process; a variable-length encoder that performs variable-lengthcoding of the compressed data and outputs coded data; an expansivedecoder that restores the compressed data to an expanded differenceimage by a prescribed expansive decoding process; a frame memory thatcontains a reproduced image; a prediction image generator that generatesa prediction image using the reproduced image stored in the frame memoryas a reference image; and an adder that generates a reproduced image byadding the expanded difference image and the prediction image, andoutputs the reproduced image and, simultaneously, stores the reproducedimage into the frame memory; wherein the prediction image generatorgenerates the prediction image using, as the reference image, at leastone of two reproduced signals which have been recently reproduced, whichone has significant image data to be referred to. Therefore, in the casewhere plural objects constituting an image are subjected to compressivecoding and transmitted object by object, when a prediction image isgenerated using forward and backward reference images, it is avoidedthat variable-size images which have already disappeared are used as thereference images, resulting in a predictive coding apparatus capable ofefficient compressive coding that suppresses the code quantity.

According to a thirteenth aspect of the present invention, there isprovided an image predictive coding apparatus comprising input means towhich data of a variable-size image is input, which data is divided intounits subjected to coding; a subtracter that obtains a difference imagebetween an object image being an object of coding and a prediction imagecorresponding to the object image; a compressive encoder that convertsthe difference image to compressed data by a prescribed compressivecoding process; a variable-length encoder that performs variable-lengthcoding of the compressed data and outputs coded data; an expansivedecoder that restores the compressed data to an expanded differenceimage by a prescribed expansive decoding process; a frame memory thatcontains a reproduced image; a prediction image generator that generatesa prediction image using the reproduced image stored in the frame memoryas a reference image; an adder that generates a reproduced image byadding the expanded difference image and the prediction image, andoutputs the reproduced image and, simultaneously, stores the reproducedimage into the frame memory; and a shape detector that detects whetherthe reproduced image includes significant image data to be referred toor not, on the basis of shape data showing the shape of an object andincluded in the variable-size image data; wherein the prediction imagegenerator receives an output from the shape detector and, when thereproduced image has no significant image data, the prediction imagegenerator generates the prediction image using, as a reference image, atleast one reproduced image which has been recently reproduced andincludes significant image data. In this apparatus, when it is detectedby the shape detector that the input shape signal has a shape, the shapesignal is subjected to predictive coding and, when the input shapesignal has no shape, the shape signal is not subjected to predictivecoding. Therefore, when plural objects constituting an image aresubjected to compressive coding and transmitted object by object, it isavoided that a variable-size image which has already disappeared is usedas a reference image for predictive coding, resulting in a predictivecoding apparatus capable of efficient compressive coding that suppressesthe code quantity.

According to a fourteenth aspect of the present invention, there isprovided a data storage medium that contains a program for implementinga predictive decoding process by a computer, wherein the program isconstructed so that the computer executes an image predictive decodingprocess according to any of the above-described image predictivedecoding apparatuses. Therefore, it is possible to realize, by software,a predictive decoding process that can decode coded data obtained byefficient compressive coding that suppresses the code quantity.

According to a fifteenth aspect of the present invention, there isprovided a data storage medium that contains a program for implementinga predictive coding process by a computer, wherein the program isconstructed so that the computer executes an image predictive codingprocess according to any of the above-described image predictive codingapparatuses. Therefore, it is possible to realize, by software, apredictive coding process capable of efficient compressive coding thatsuppresses the code quantity.

According to a sixteenth aspect of the present invention, there isprovided a data storage medium that contains a program for implementinga predictive coding process by a computer, wherein the program isconstructed so that the computer executes an image predictive codingprocess according to any of the above-described image predictive codingapparatuses. Therefore, it is possible to realize, by software, apredictive coding process capable of efficient compressive coding thatsuppresses the code quantity.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flowchart of a prediction image generation process in animage predictive decoding method according to a first embodiment of thepresent invention.

FIGS. 2(a) and 2(b) are schematic diagrams for explaining imageprediction in the image predictive decoding method according to thepresent invention.

FIG. 3 is a block diagram illustrating an image predictive decodingapparatus according to the first embodiment of the invention.

FIG. 4 is a block diagram illustrating a frame memory unit used in theimage predictive decoding apparatus according to the first embodiment ofthe invention.

FIG. 5 is a flowchart of a prediction image generation process in animage predictive decoding method according to a third embodiment of thepresent invention.

FIG. 6 is a flowchart of a prediction image generation process in animage predictive decoding method according to a fourth embodiment of thepresent invention.

FIG. 7 is a diagram showing image data according to the first embodimentof the invention.

FIG. 8 is a flowchart of a prediction image generation process in animage predictive decoding method according to a second embodiment of theinvention.

FIG. 9 is a diagram showing image data according to the secondembodiment of the invention.

FIG. 10 is a flowchart of a prediction image generation process in animage predictive decoding method according to a fifth embodiment of thepresent invention.

FIG. 11 is a flowchart of a prediction image generation process in animage predictive decoding method according to a sixth embodiment of thepresent invention.

FIG. 12 is a block diagram illustrating an image predictive codingapparatus according to a seventh embodiment of the present invention.

FIG. 13 is a block diagram illustrating an image predictive codingapparatus according to an eighth embodiment of the present invention.

FIGS. 14(a)-14(c) are diagrams for explaining a data storage mediumwhich contains a program for implementing image processing by acomputer, which image processing is one of the methods and apparatusesaccording to the first to eighth embodiments of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS EMBODIMENT 1

FIG. 1 is a flowchart of a prediction image generation process in animage predictive decoding method according to a first embodiment of thepresent invention. Before explaining FIG. 1, an image prediction methodaccording to this first embodiment will be described using FIGS. 2(a)and 2(b).

The size of an input image used in the image predictive decoding methodof this first embodiment is variable, and it may happen that the sizebecomes zero.

FIG. 2(a) shows images 201˜210 of a motion picture, which are arrangedin the display order. The image 201 is the first frame to be displayed,followed by 202, 203, . . . , and this order is shown by #1˜#10. Sincethe image #1 (201) is the first image, it is subjected to intra-framecoding. In this first embodiment, an image (one frame) is divided intoplural blocks each having the size of 8×8 pixels, and each block of 8×8pixels is subjected to DCT and quantization. The quantized coefficientis subjected to variable-length coding. In decoding, the coded dataobtained by the variable-length coding is subjected to variable-lengthdecoding, and the quantized coefficient obtained by the variable-lengthdecoding is subjected to inverse quantization and inverse DCT, therebyreproducing the image. Next, the image #2 (202) is subjected tointer-frame predictive coding by referring to the reproduced image #1(201).

In this first embodiment, using block matching as a motion detectionmethod, a prediction block having the smallest error from the objectblock currently being processed is detected from the image #1 (201). Onthe basis of the detected motion from the object block toward theprediction block, an optimum prediction block is obtained by motioncompensation of the object block from the reproduced image #1 (201).Next, a difference between the object block and the correspondingprediction block is obtained, and the difference is subjected to DCT.The DCT coefficient is quantized, and the quantized output istransmitted or stored together with the motion information. Thereproduced image #1 (201) serves as a reference image for the image #2(202). This prediction is called “forward prediction”. In decoding, theprediction block is added to the difference which has been subjected toinverse quantization and inverse DCT, thereby reproducing the image.

In like manner, the image #3 (203) and the image #4 (204) are subjectedto predictive coding using reference images shown by the arrows. Likethe images #6 (206), #8 (208) and #10 (210), prediction may be carriedout from a previous image but one. In contrast with the forwardprediction, like the images #5 (205), #7 (207) and #9 (209), predictionmay be carried out by referring to a future image to be displayed afterthe object image. This prediction is called “backward prediction”. Whenboth the forward prediction and the backward prediction are carried out,this is called “bidirectional prediction”. The bidirectional predictionhas three modes: forward prediction mode, backward prediction mode,interpolation mode for balancing the forward prediction and the backwardprediction.

FIG. 2(b) shows the transmission order, i.e., decoding order, of theimages predicted as shown in FIG. 2(a).

The image #1 (211) is initially decoded and reproduced. Referring to thereproduced image #1, the image #2 (212) is decoded. With respect to thebidirectional prediction image like the image #5 (216), the referenceimages used for the prediction have to be decoded and reproduced beforethe prediction image. Therefore, the image #6 (215) is decoded beforethe image #5 (216). Likewise, the image #8 (217) and the image #10 (219)are transmitted, decoded and reproduced before the image #7 (218) andthe image #9 (220), respectively.

When transmitting a variable-size image, the size of the image must betransmitted. In this first embodiment, the image size is described atthe head of the coded data of the image, and the horizontal and verticalsizes Hm and Vm are shown by 20 bits each. FIG. 7 shows coded image data(VD) according to this first embodiment, and the coded data includes themotion vector, the quantization width, and the DCT coefficient, inaddition to the horizontal and vertical sizes Hm and Vm.

Next, a description is given of the prediction image generation processin the image predictive decoding method according to the firstembodiment.

In order to generate a prediction image, initially, the size of theprevious reference image is input (step 102), and it is examined whetherthe size of the reference image is zero or not (step 103).

In the decoding order shown in FIG. 2(b), a reference image alwaysexists before an image being an object of decoding (in coding, an objectof coding). That is, the reference image is a most recently reproducedimage in the predictive decoding method of this first embodiment. Forexample, in FIG. 2(b), a reference image for the image #4 (214) is theimage #3 (213). However, an image reproduced by bidirectional predictioncannot be used for prediction, so that this image cannot be a referenceimage. Therefore, for example, a reference image for the image #8 (217)is the image #6 (215).

When it is decided in step 103 that the size of the reference mage isnot zero, step 104 follows, wherein a prediction image is generatedusing the reference image. On the other hand, when it is detected instep 103 that the size of the reference image is zero, step 105 follows,wherein a prediction image is generated using, as a reference image, arecently reproduced image of which size is not zero. The way ofdetecting a recently reproduced image of which size is not zero will bedescribed hereinafter using FIG. 2(b).

In the case of generating a prediction image of the image #4 (214), itis assumed that the size of the image #3 (213) just before the image #4(214) is zero, and the size of the image #2 is not zero. In this case, aprediction image of the image #4 (214) is generated by referring to theimage #2 (212). Likewise, in the case of generating a prediction imageof the image #6 (215), assuming that the sizes of the images #3 (213)and #4 (214) are zero, the prediction image is generated by referring tothe image #2 (212). This first embodiment employs block by block motioncompensation as a method for generating a prediction image, like MPEG1.

FIG. 3 is a block diagram illustrating an image predictive decodingapparatus 300 according to the first embodiment of the invention.

The image predictive decoding apparatus 300 receives image data obtainedby compressively coding a variable-size image by a prescribed method,and performs predictive decoding of the image data.

The image predictive decoding apparatus 300 includes a data analyzer302, a decoder 303, and an adder 306. The data analyzer 302 analyzes thecompressively coded image data, and outputs the quantization width andthe DCT coefficient to the line 312, the motion vector to the line 318,and the image size to the line 321. The decoder 303 transforms thecompressed block data (compressed block) from the data analyzer 302 toan expanded block by data expansion. The adder 306 adds the expandedblock and the prediction block to generate a reproduced block.

Further, the image predictive decoding apparatus 300 includes a framememory unit 309 and a prediction image generator 310. The frame memoryunit 309 stored the reproduced block. The prediction image generator 310generates an address for accessing the frame memory unit 309 on thebasis of the motion vector and obtains, as a prediction block, a blockcorresponding to the address from the image stored in the frame memoryunit 309. In this first embodiment, the prediction image generator 310decides, as a reference image, a single reproduced image which has beenrecently reproduced and includes significant image content data to bereferred to, on the basis of the image size from the data analyzer 302.The decision of a reference image may be carried out, as shown by dottedlines in FIG. 3, by using a controller 320 that controls the framememory unit 309 according to the image size from the data analyzer 302.That is, the frame memory unit 309 is controlled by the controller 320so as to select a single reproduced image which has been recentlyreproduced and includes significant image data to be referred to.

The decoder 303 comprises an inverse quantizer 304 that inverselyquantizes the compressed block from the data analyzer 302, and aninverse discrete cosine transformer (hereinafter referred to as IDCT)305 that performs inverse DCT (transformation of a frequency regionsignal to a spatial region signal) to the output from the inversequantizer 304.

Further, reference numerals 301 and 307 designate an input terminal andan output terminal of the image predictive decoding apparatus 300.

A description is given of the operation of the image predictive decodingapparatus shown in FIG. 3.

First of all, image data (coded data) obtained by compressively coding avariable-size image in a prescribed method is input to the inputterminal 301. In this first embodiment, compressive coding is carriedout using motion compensation DCT as in MPEG1, so that the coded dataincludes the motion vector, quantization width, DCT coefficient, anddata of image size.

Next, in the data analyzer 302, the compressively coded image data isanalyzed, and the quantization width and the DCT coefficient aretransmitted, as compressed block data, through the line 312 to thedecode 303. Further, the motion vector analyzed in the data analyzer 302is transmitted through the line 318 to the prediction image generator310. Likewise, the image size analyzed by the data analyzer 302 istransmitted through the like 321 to the controller 320.

In the decoder 303, the compressed block data, i.e., compressed block,are expanded by the inverse quantizer 304 and the inverse DCTtransformer 305, thereby generating an expanded block 314. To bespecific, the inverse quantizer 304 inversely quantizes the compressedblock, and the inverse DCT transformer 305 transforms the frequency areasignal to the spatial area signal, thereby generating the expanded block314. In the prediction image generator 310, according to the motionvector transmitted through the line 318, an address 321 for accessingthe frame memory unit 309 is generated, and this address 321 is input tothe frame memory unit 309. Then, a prediction block 317 is generatedfrom images stored in the frame memory unit 309. The prediction block317 (319) and the expanded block 314 are input to the adder 306, whereinthese blocks 319 and 314 are added, thereby generating a reproducedblock 315. The reproduced block 315 is output from the output terminal307 and, simultaneously, it is transmitted through the line 316 andstored in the frame memory unit 309. When intra-frame decoding iscarried out, the sample values of the prediction block are all zero.

The operation of the prediction image generator 310 is identical to thatalready described with respect to the flowchart of FIG. 1. That is, thesize of the reference image is input to the prediction image generator310, and the reference image is decided in the prediction imagegenerator 310. The decision of the reference image may be carried out bycontrolling the frame memory unit 309 according to information whetherthe size of the reference image is zero or not, which information istransmitted through the controller 320 and the line 322.

FIG. 4 is a block diagram illustrating a frame memory bank 406 as anexample of the frame memory unit 309 in the image predictive decodingapparatus 300 according to the first embodiment. The frame memory bank406 includes three frame memories 401˜403. The reproduced image isstored in one of the frame memories 401˜403. When generating aprediction image, these frame memories 401˜403 are accessed.

In this first embodiment, the frame memory bank 406 has switches 404 and405. The switch 405 is to select a frame memory for storing thereproduced image which is input through the line 408 (corresponding tothe line 316 in FIG. 3), from the frame memories 401˜403. The switch 405selects the frame memories 401˜403 one by one, controlled by thecontroller 320, i.e., according to the control signal 322. That is,after the first reproduced image is stored in the frame memory 401, thesecond reproduced image is stored in the frame memory 402. After thethird reproduced image is stored in the frame memory 403, the switch 405selects the frame memory 401. The switch 404 is connected through theline 407 (corresponding to the line 317 in FIG. 3) to the predictionimage generator 310. Also this switch 404 selects the frame memories401˜403 one by one, controlled by the controller 320, i.e., according tothe control signal 322. However, the switching order is changedaccording to the size of the reference image. For example, although theswitch 404 is to be connected to the frame memory 402 for generation ofa prediction image according to the given order, when the image size ofthe frame memory 402 is zero, the controller 320 controls the switch 404so as to select the previous frame memory 401 (on the premise that theimage size of the frame memory 401 is not zero). In this way, aprediction image can be generated from a reference image of which sizeis not zero. The switch 404 may be connected to plural frame memories atthe same time. Further, in a unit where each frame memory is reset atevery reproduction of a single image, a recently reproduced image ofwhich size not zero can be left in the frame memory by controlling theunit with the controller 320 so that the frame memory is not reset whenthe size of the reproduced image is zero. In other words, it is possibleto prevent the frame memory from being updated.

While in this first embodiment the block by block motion compensationDCT method is described, the present invention is applicable to otherprediction methods using, for example, global motion compensation orarbitrary lattice-shaped block motion compensation. Further, although inthis first embodiment a prediction image is generated from a singlereproduced image serving as a reference image, the present invention issimilarly applicable to the case where a prediction is generated fromplural reference images.

As described above, according to the first embodiment of the invention,the size of a previous reference image which is input to the apparatusis detected and, when the size of the reference image is not zero, aprediction image is generated using the reference image. On the otherhand, when the size of the previous reference image is zero, aprediction image is generated using a recently reproduced image of whichsize is not zero. Therefore, when plural objects constituting an imageare subjected to compressive coding and transmitted object by object toincrease the compression efficiency, it is avoided that a variable-sizeimage which has already disappeared is used as a reference image forpredictive decoding or coding, resulting in appropriate predictivedecoding or coding capable of suppressing the residual signal(difference signal). Further, the coded data obtained by the imagepredictive coding apparatus according to this seventh embodiment can bedecoded correctly by the image predictive decoding apparatus accordingto the second embodiment.

Embodiment 2

In the first embodiment of the invention, it is detected whether thesize of the reference image is zero or not, and the reference image isdecided using the detected information. However, when the fact that theimage size is zero is shown by another index (e.g., one-bit flag F),control can be carried out using this index. In this second embodimentof the invention, generation of a prediction image is controlled usingsuch index.

That is, in this second embodiment, as shown in FIG. 9, coded data of anobject image includes a one-bit flag F showing that the image size iszero, i.e., the corresponding reference image is completely transparentand has no coded image content data, and this flag F is placed beforethe horizontal and vertical sizes Hm and Vm showing the image size. Whenthe image size is zero, the flag F is “0”. In this second embodiment,generation of a prediction image is controlled using the flag F.

Hereinafter, a description is given of a prediction image generationprocess in the image predictive decoding method according to the secondembodiment, using the flowchart of FIG. 8.

To generate a prediction image, initially, a previous reference image isinput in step 802, and it is examined in step 803 whether the flag F ofthe reference image is “1” or not. When it is decided in step 803 thatthe flag F of the reference image is “1”, the size of this referenceimage is not zero, namely, the reference image is not completelytransparent and has coded image content data. So, in step 804, aprediction image is generated using the previous reference image.

When it is decided in step 803 that the flag F of the reference image isnot “1”, step 805 follows, wherein a prediction image is generatedusing, as a reference image, a recently reproduced image of which flag Fis not “0”.

As described above, according to the second embodiment of the invention,when plural objects constituting an image are subjected to compressivecoding and transmitted object by object, it is avoided that avariable-size image which has already disappeared is used as a referenceimage, resulting in appropriate predictive coding or coding capable ofsuppressing the residual signal (difference signal). In addition, thecoded data of the object image has, at its head, a flag showing whetheror not the previously reproduced image has significant coded imagecontent data to be referred to, and the reference image is decided bydetecting this flag. So, the operation of deciding the reference imageis facilitated.

Embodiment 3

FIG. 5 is a flowchart of a prediction image generation process in animage predictive decoding method according to a third embodiment of thepresent invention. The prediction image generation process according tothis third embodiment is fundamentally identical to that according tothe first embodiment except that step 505 in FIG. 5 takes the place ofstep 105 in FIG. 1. In step 505, when the reference image is zero orwhen the reference image is completely transparent (or when the flag Fof the image is “0”), a prediction image to which a prescribed value isassigned, i.e., a prediction image having a prescribed value, isgenerated.

In this third embodiment, it is assumed that the prediction image isgray, i.e., both the luminance signal value and the color differencesignal value thereof are 128. As a result, when coding, the gray blockis subtracted from the block being an object of coding. When decoding,the gray block is added to the block being an object of decoding. Theprescribed value mentioned above may be variable, and this value may betransmitted from the encoder to the decoder to be used for generating aprediction image.

As described above, according to the third embodiment of the invention,when plural objects constituting an image are subjected to compressivecoding and transmitted object by object, it is avoided that avariable-size image which has already disappeared is used as a referenceimage, resulting in appropriate predictive decoding or coding capable ofsuppressing the residual signal (difference signal). Further, when thesize of the reference image is zero, i.e., when the reference image iscompletely transparent, a prediction image having a prescribed value isgenerated. Therefore, in addition to the same effects as provided by thefirst embodiment, generation of the prediction image is facilitated.

Embodiment 4

FIG. 10 is a flowchart of a prediction image generation process in animage predictive decoding method according to a fourth embodiment of thepresent invention. The prediction image generation process according tothis fourth embodiment is fundamentally identical to that according tothe second embodiment except that step 1005 in FIG. 10 takes the placeof step 805 in FIG. 8. In step 1005, when the flag F of the referenceimage is “0”, a prediction image to which a prescribed value isassigned, i.e., a prediction image having a prescribed value, isgenerated.

According to the fourth embodiment of the invention, when plural objectsconstituting an image are subjected to compressive coding andtransmitted object by object, it is avoided that a variable-size imagewhich has already disappeared is used as a reference image, resulting inappropriate predictive decoding or coding capable of suppressing theresidual signal (difference signal). Further, the coded data of theobject image has, at its head, a flag showing whether or not thepreviously reproduced image has significant coded data to be referredto, and when it is detected that this flag is “0”, a prediction imagehaving a prescribed value is generated. Therefore, in addition to thesame effects as provided by the second embodiment, generation of theprediction image is facilitated.

Embodiment 5

FIG. 6 is a flowchart of a prediction image generation process in animage predictive decoding method employing bidirectional prediction,according to a fifth embodiment of the present invention. Hereinafter, adescription is given of the bidirectional prediction process in the casewhere the reference image size is zero, i.e., when the reference imageis completely transparent.

Initially, in step 602, the sizes of forward and backward referenceimages are input. The image #5 (205), a first frame, shown in FIG. 2(a)is a bidirectional prediction image of which forward reference image andbackward reference image are the image #4(204), a second frame, and theimage #6 (206), a third frame, respectively.

When it is decided in steps 603 and 604 that the sizes of both theforward and backward reference images are zero, an image to which aprescribed value is assigned, i.e., an image having a prescribed value,is generated as a prescribed value in step 605.

When it is decided in steps 603 and 604 that the size of the forwardreference image is zero and the size of the backward reference image isnot zero, a prediction image is generated using only the backwardreference image in step 606.

When it is decided in steps 603 and 607 that the size of the forwardreference image is not zero and the size of the backward reference imageis zero, a prediction image is generated using only the forwardreference image in step 608.

When it is decided in steps 603 and 607 that the sizes of both theforward and backward reference images are not zero, a prediction imageis generated using these reference images.

In step 610, the generated prediction image is output. Receiving theprediction image, the encoder subtracts the prediction image from theobject image, while the decoder adds the prediction image to thedifference of the object image. In this way, the residual signal(difference signal) can be suppressed.

As described above, according to the fifth embodiment of the invention,in the case where plural objects constituting an image are subjected tocompressive coding and transmitted object by object, when a predictionimage is generated using forward and backward reference images, it isavoided that variable-size images which have already disappeared areused as the reference images, resulting in appropriate predictivedecoding or coding capable of suppressing the residual signal(difference signal). Further, since a prediction image having aprescribed value is generated, generation of the prediction image isfacilitated.

Embodiment 6

FIG. 11 is a flowchart of a prediction image generation process in apredictive decoding method using bidirectional prediction, according toa sixth embodiment of the present invention. This sixth embodiment isfundamentally identical to the fifth embodiment, in like manner that thesecond and fourth embodiments are fundamentally identical to the firstand third embodiments, respectively. To be specific, in this sixthembodiment, “size is zero ?” in steps 603, 604 and 607 in FIG. 6 arechanged to “flag F is 0 ?” as shown in steps 1103, 1104 and 1107 in FIG.11.

According to the sixth embodiment of the invention, in the case whereplural objects constituting an image are subjected to compressive codingand transmitted object by object, when a prediction image is generatedusing forward and backward reference images, it is avoided thatvariable-size images which have already disappeared are used as thereference images, resulting in appropriate predictive decoding or codingcapable of suppressing the residual signal (difference signal). Further,when it is detected that the first and second flags F of the forward(second) and backward (third) reference images, respectively, are “0”, aprediction image having a prescribed value is generated. Therefore,detection of the variable-size image which has already disappeared isfacilitated, and generation of the prediction image is facilitated.

Embodiment 7

FIG. 12 is a block diagram illustrating an image predictive codingapparatus 1000 according to a seventh embodiment of the presentinvention. The coding apparatus 1000 comprises a texture coding unit1100 that performs predictive coding of a texture signal comprising aluminance signal and a color difference signal, and a shape coding unit1200 that performs predictive coding of a shape signal.

The texture coding unit 1100 comprises a blocking unit 1110 that dividesa texture signal per frame into plural macroblocks each having the sizeof 16×16 pixels (a unit subjected to coding) and outputs the dividedtexture signal; a subtracter 1160 that calculates a difference between ablock being an object of coding (hereinafter, referred to as an objectblock) and a prediction block corresponding to the object block; acompressive encoder 1120 that compressively codes the difference; and alocal decoder 1130 that expansively decodes the output from thecompressive encoder 1120. The compressive encoder 1120 comprises adiscrete cosine transformer (hereinafter referred to as a DCT) 1121 thatperforms discrete cosine transformation (DCT) of the difference, and aquantizer 1122 that quantizes the DCT coefficient. The local decoder1130 comprises an inverse quantizer 1131 that inversely quantizes theoutput from the quantizer 1122, and an inverse discrete cosinetransformer (hereinafter referred to as an IDCT) 1132 that performsinverse DCT (transformation of a frequency region signal to a spatialregion signal) to the output from the inverse quantizer 1131.

Further, the texture coding unit 1100 includes an adder 1170 that addsan expanded block output from the IDCT 1132 and the prediction block togenerate a reproduced block; a frame memory unit (FM1) 1140 that storesthe reproduced block; and a prediction image generator 1150 that obtainsa prediction block corresponding to the object block from images storedin the frame memory unit 1140 by motion compensation on the basis ofmotion information detected by a prescribed motion detection method.

The prediction image generator 1150 decides a reference image to bereferred to when generating a prediction block (prediction image) fromthe reproduced images stored in the frame memory unit 1140 on the basisof the image size obtained from the output of the blocking unit 1110.

On the other hand, the shape coding unit 1200 comprises a blocking unit1210 that divides a shape signal per frame into plural macroblocks eachhaving the size of 16×16 pixels (a unit subjected to coding) and outputsthe divided shape signal; a subtracter 1260 that calculates a differencebetween a block being an object of coding (object block) and aprediction block corresponding to the object block; a shape encoder 1220that codes the difference by a prescribed coding method; and a shapedecoder 1230 that decodes the output from the shape encoder 1220 by adecoding method corresponding to the coding method. The shape encoder1120 codes the output from the subtracter 1260 by a coding method suchas quarter tree or chain coding.

The shape coding unit 1200 further comprises an adder 1270 that adds adecoded block output from the shape decoder 1230 and the predictionblock to generate a reproduced block; a frame memory unit (FM2) 1240that stores the decoded block output from the adder 1270; and aprediction image generator 1250 that obtains a prediction blockcorresponding to the object block from the shape information stored inthe frame memory unit 1240 by motion compensation based on the motioninformation detected by a prescribed motion detection method.

Further, the prediction image generator 1250 decides a reference imageto be referred to when generating a prediction block (prediction image),from the reproduced images stored in the frame memory unit 1240, on thebasis of the image size obtained from the output of the blocking unit1210.

The decision of the reference image by the coding unit 1100 or 1200 maybe carried out, as shown by dotted lines in FIG. 12, by using a shapedetector 1280 that performs shape detection on the basis of thereproduced block, and controlling the frame memory units 1140 and 1240according to the result of shape detection which is output from theshape detector 1280. In this case, the control of the frame memory unitsaccording to the result of shape detection is identical to the controlof the frame memory unit 309 by the controller 320 according to thefirst embodiment. Further, the result of shape detection is applied to avariable-length encoder 1010 which is described later, and transmittedtogether with coded data of the texture signal and the shape signal.

Further, the image predictive coding apparatus 1000 includes avariable-length encoder 1010. The variable-length encoder 1010 performsvariable-length coding of the coded texture signal output from thetexture encoder 1100 and the coded shape signal and the result of shapedetection, which are output from the shape encoder 1200, and multiplexesthese signals to be output.

In FIG. 12, reference numeral 1001 denotes an input terminal for thetexture signal, 1002 denotes an input terminal for the shape signal, and1003 denotes an output terminal for the coded data.

A description is given of the operation.

When a texture signal (luminance/color-difference signals) and a shapesignal are input to the image predictive coding apparatus 1000, thetexture signal and the shape signal are divided into macroblocks (unitssubjected to coding) by the blocking units 1110 and 1210 included in thecoding units 1100 and 1200, respectively, the prediction coding iscarried out for each macroblock.

In the texture signal coding unit 1100, the subtracter 1160 calculates adifference between an object block and a prediction block, the DCT 1121transforms this difference to a DCT coefficient, and the quantizer 1122quantizes the DCT coefficient to generate a quantized coefficient. Thequantized coefficient is output toward the variable-length encoder 1010.

The inverse quantizer 1131 inversely quantizes the quantized coefficientto generate a DCT coefficient, and the IDCT 1130 transforms the DCTcoefficient to an expanded block corresponding to the object block by aprocess of transforming frequency region data to spatial region data.Further, the adder 1170 adds the expanded block and the prediction blockto generate a reproduced block. The reproduced block is stored in theframe memory unit 1140. At this time, the prediction image generator1150 generates a prediction block corresponding to the object block,from the images stored in the frame memory unit 1140, by motioncompensation on the basis of motion information detected by a prescribedmotion detection method. Further, the prediction image generator 1150decides, as a reference image, a single reproduced image which has beenrecently reproduced and includes significant image data to be referredto, from the reproduced images stored in the frame memory unit 1140.When the apparatus is provided with the shape detector 1280, thedecision of the reference image can be performed by controlling theframe memory until 1140 according to the output from the shape detector1280, i.e., information whether the size of the reproduced image to bereferred to is zero or not.

In parallel with the processing of the texture encoder 1100, in theshape encoder 1200, predictive coding of the shape signal is carried outin similar manner to the above-described predictive coding of thetexture signal. That is, a difference between the object block and theprediction block is obtained by the subtracter 1260, and this differenceis coded by a coding method such as quarter tree or chain coding in theshape encoder 1220, and the coding result is output toward thevariable-length encoder 1010. Further, the coded shape signal from theshape encoder 1220 is restored by the shape decoder 1230, and therestored block and the prediction block are added by the adder 1270 togenerate a reproduced block.

The reproduced block output from the adder 1270 is stored in the framememory unit 1240. In the prediction image generator 1250, a predictionblock corresponding to the object block is generated from the shapeinformation stored in the frame memory unit 1240, by motion compensationbased on motion information detected by a prescribed motion detectionmethod. Further, in the prediction image generator 1250, a singereproduced image which has been recently reproduced and includessignificant image data to be referred to is decided as a referenceimage, from the reproduced images stored in the frame memory unit 1240,on the basis of the image size obtained from the output of the blockingunit 1210.

When the apparatus is provided with the shape detector 1280, thedecision of the reference image can be performed by controlling theframe memory unit 1240 according to the output from the shape detector1280, i.e., information whether the size of the reproduced image to bereferred to is zero or not. In this case, the reproduced block is inputto the shape detector 1280 wherein shape detection is carried out. Forexample, assuming that the shape signal is a binary signal, when thereis only black data between white data and black data as shape data, noreproduced data exists. At this time, there is no texture signalcorresponding to the shape signal of this block. In this case, asdescribed above, a flag showing “no coded data ” or data showing “imagesize is zero” is output from the shape detector 1280 toward the framememory units 1140 and 1240 and the variable-length encoder 1010. In theframe memory units 1140 and 1240, according to the output from the shapedetector 1280, control is carried out in similar manner to the controlof the frame memory unit 309 by the controller 320 according to thefirst embodiment.

As described above, according to the seventh embodiment of the presentinvention, in the coding unit 1120 (1200), a single reproduced signalwhich has been recently reproduced and includes significant image datato be referred to is decided as a reference image from the reproducedimages stored in the frame memory unit 1140 (1210) according to theimage size obtained from the output of the blocking unit 1110 (1210).Therefore, when plural objects constituting an image are subjected tocompressive coding and transmitted object by object, it is avoided thata variable-size image which has already disappeared is used as areference image for predictive coding, whereby appropriate predictivecoding that can suppress the residual signal (difference signal) iscarried out. Further, the coded data obtained by the image predictivecoding apparatus according to this seventh embodiment can be decodedcorrectly by the image predictive decoding apparatus according to thesecond embodiment.

Further, when the apparatus includes the shape detector 1280, thedecision whether a reference image corresponding to the input objectblock exists or not is performed by detecting the shape of thereproduced block of the shape signal, in the shape coding unit 1200.When the reproduced block has no shape, in the texture encoder and theshape encoder, a prediction block is generated using a reproduced blockwhich has been recently reproduced and has a shape, instead of thereproduced block corresponding to the object block. Therefore, whenplural objects constituting an image are subjected to compressive codingand transmitted object by object, it is avoided that a variable-sizeimage which has already disappeared is used as a reference image forpredictive coding, whereby appropriate predictive coding is carried out.Also in this case, the coded data obtained by the image predictivecoding apparatus according to this seventh embodiment can be decodedcorrectly by the image predictive decoding apparatus according to thesecond embodiment. That is, in the image predictive decoding apparatus,the data analyzer 302 controls the frame memory unit 309 on the basis ofthe output from the shape detector 1280. Therefore, when coded dataobtained by object by object predictive coding is decoded, it is avoidedthat a variable-size image which has already disappeared is used as areference image for predictive decoding, whereby appropriate predictivedecoding is carried out.

In this seventh embodiment, the selection of the reproduced image as areference image by the prediction image generator 1150 (1250) or thecontrol of the frame memory unit 1140 (1240) according to the result ofshape detection is carried out in the same manner as the selection ofthe reproduced image as a reference image by the prediction imagegenerator 1150 (1250) or the control of the frame memory unit 309 by thecontroller 320 according to the first embodiment, respectively. However,the present invention is not restricted thereto.

For example, when there is no image data to be referred to in a frameprevious to the object frame, a prediction image having a prescribedvalue may be generated as described for the third embodiment. In thiscase, as an image predictive decoding apparatus corresponding to theimage predictive coding apparatus, an apparatus that performs the imagepredictive decoding process according to the third embodiment isemployed.

Further, the prediction according to this seventh embodiment may bebidirectional prediction as described for the fifth embodiment. In thiscase, as an image predictive decoding apparatus corresponding to theimage predictive coding apparatus, an apparatus that performs the imagepredictive decoding process according to the fifth embodiment isemployed.

Embodiment 8

FIG. 13 is a block diagram illustrating an image predictive codingapparatus 1000 a according to an eighth embodiment of the presentinvention. The coding apparatus 1000 a comprises a texture coding unit1100 a that performs predictive coding of a texture signal comprising aluminance signal and a color difference signal, and a shape coding unit1200 a that performs predictive coding of a shape signal.

The texture coding unit 1100 a is different from the texture coding unit1100 according to the seventh embodiment only in that a switch 1190 isconnected between the input terminal 1001 and the blocking unit 1110,which switch connects (supplies) the texture signal to either of theblocking unit 1110 and the ground, according to a control signal.

The shape coding unit 1200 a is different from the shape coding unit1200 according to the seventh embodiment only in that it does notinclude the shape detector 1280, and a switch 1290 is connected betweenthe input terminal 1002 and the blocking unit 1210, which switchconnects (supplies) the shape signal to either of the blocking unit 1210and the ground, according to a control signal.

The image predictive coding apparatus 1000 a further includes a shapedetector 1020 that receives the shape signal and outputs the result ofshape detection toward the switches 1190 and 1290 as the control signal.When it is detected by the shape detector 1020 that the input shapesignal has no shape, the switch 1190 (1290) connects the texture signal(shape signal) to the ground. Conversely, when the input shape signalhas a shape, the switch 1190 (1290) connects the texture signal (shapesignal) to the blocking unit 1110 (1210). The result of shape detectionis subjected to variable-length coding by the variable-length encoder1010, together with the coded data from the coding units 1100 a and 1200a.

A description is now given of the operation of the image predictivecoding apparatus 1000 a according to this eighth embodiment. Theoperation of the apparatus 1000 a is identical to the operation alreadydescribed for the seventh embodiment except that the switches 1190 and1290 are controlled by the shape detector 1020.

To be specific, when the texture signal and the shape signal are input,the shape detector 1020 detects whether the input shape signal has ashape or not. When the shape signal does not have a shape, the switches1190 and 1290 are controlled by the output from the shape detector 1020so that the texture signal and the shape signal are supplied to theground. That is, at this time, the texture signal and the shape signalare not subjected to predictive coding, and the result of shapedetection by the shape detector 1020 is supplied to the variable-lengthencoder 1010.

On the other hand, when it is detected that the input shape signal has ashape, the switches 1190 and 1290 are controlled by the output from theshape detector 1020, and the texture signal and the shape signal areinput to the blocking units 1110 and 1210, respectively, wherein thesignals are subjected to predictive coding. The result of shapedetection by the shape detector 1020 is supplied to the variable-lengthencoder 1010, together with the outputs from the coding units 1100 a and1200 a.

As described above, according to the eighth embodiment of the presentinvention, the image predictive coding apparatus includes the shapedetector 1020 that detects whether the input shape signal has a shape ornot. When the shape signal has a shape, the texture signal and the shapesignal are subjected to predictive coding, and when the shape signaldoes not have a shape, the texture signal and the shape signal are notsubjected to predictive coding. Therefore, when plural objectsconstituting an image are subjected to compressive coding andtransmitted object by object, it is avoided that a variable-size imagewhich has already disappeared is used as a reference image forpredictive coding, whereby appropriate predictive coding that cansuppress the residual signal (difference signal) is carried out.

Further, since the result of shape detection by the shape detector 1020is coded and transmitted, an image predictive decoding apparatus thatreceives the result of shape detection can appropriately performprediction decoding of a variable-size image that has alreadydisappeared, using the result of shape detection as a synchronoussignal. That is, while the variable-size image disappears, reproductionof coded data corresponding to this image is stopped.

Furthermore, when a program for implementing the image predictivedecoding method (apparatus) or the image predictive coding method(apparatus) according to any of the aforementioned embodiments of theinvention is recorded in a storage medium such as a floppy disk, theimage processing according to the embodiment can be executed easily inan independent computer system.

FIGS. 14(a)-14(c) are diagrams for explaining the case where the imagepredictive decoding process of the image predictive coding processaccording to any of the aforementioned embodiments is executed by acomputer system using a floppy disk which contains a programcorresponding to the process.

FIG. 14(a) shows a front view of a floppy disk FD, a cross-sectionalview thereof, and a floppy disk body D as a storage medium. FIG. 14(b)shows an example of a physical formation of the floppy disk body D. Thefloppy disk body D is contained in a case FC. On the surface of the diskbody D, a plurality of tracks Tr are formed concentrically from theouter circumference of the disk toward the inner circumference. Eachtrack is divided into 16 sectors in the angular direction. Therefore, inthe floppy disk body D containing the above-mentioned program, data ofthe program are recorded on assigned regions of the floppy disk body D.

FIG. 14(c) shows the structure for recording/reproducing the programin/from the floppy disk FD, where Cs is a computer system and FDD is afloppy disk drive. When the program is recorded in the floppy disk FD,data of the program are written in the floppy disk FD from the computersystem Cs through the floppy disk drive FDD. When the above-mentionedimage predictive decoding process or the image predictive coding processis constructed in the computer system Cs from the program in the floppydisk FD, the program is read from the floppy disk FD by the floppy diskdrive FDD and transmitted to the computer system Cs.

Although in the above description emphasis has been placed on a datastorage medium containing a program for performing an image predictivedecoding process or an image predictive coding process according to anyof the aforementioned embodiments, a data storage medium containingcoded image data according to any of the aforementioned embodiments isalso within the scope of the invention.

Furthermore, although in the above description emphasis has been placedon image processing by a computer system using a floppy disk as a datastorage medium, similar image processing can be carried out using otherstorage media, such as an IC card and a ROM cassette, as long as theprogram of the image processing can be recorded in the media.

What is claimed is:
 1. An image predictive decoding method for decodinga first coded frame obtained by coding an image, comprising: determiningwhether or not a second coded frame and a third coded frame, which areto be used as a forward reference image and a backward reference image,respectively, include image content data, wherein said determination isperformed based on a first flag included in the second coded frame and asecond flag included in the third coded frame, the first flag indicatingwhether or not the second coded frame includes image content data, andthe second flag indicating whether or not the third coded frame includesimage content data; selecting, as the forward reference image, a decodedframe corresponding to the second coded frame only when the second codedframe includes image content data; selecting, as the backward referenceimage, a decoded frame corresponding to the third coded frame only whenthe third coded frame includes image content data; generating aprediction image by referring to at least one of the selected forwardreference image and the selected backward reference image; and decodingthe first coded frame data by predictive decoding using the generatedprediction image.
 2. The image predictive decoding method of claim 1,wherein the image includes an arbitrarily shaped object whose size isvariable.